Phase transition for cutting-plane approach to vertex-cover problem 
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We study the vertex-cover problem which is an NP-hard optimization problem and a prototypical 
model exhibiting phase transitions on random graphs, e.g., Erdos-Renyi (ER) random graphs. These 
phase transitions coincide with changes of the solution space structure, e.g, for the ER ensemble at 
connectivity c = e w 2.7183 from replica symmetric to replica-symmetry broken. For the vertex- 
cover problem, also the typical complexity of exact branch-and-bound algorithms, which proceed by 
exploring the landscape of feasible configurations, change close to this phase transition from "easy" 
to "hard". In this work, we consider an algorithm which has a completely different strategy: The 
problem is mapped onto a linear programming problem augmented by a cutting-plane approach, 
hence the algorithm operates in a space outside the space of feasible configurations until the final 
step, where a solution is found. Here we show that this type of algorithm also exhibits an "easy- 
hard" transition around c = e, which strongly indicates that the typical hardness of a problem is 
fundamental to the problem and not due to a specific representation of the problem. 



NP-hard combinatorial optimization problems [l., 2] 
are fundamental to computational complexity, because 
despite much effort no algorithm has been found so far, 
which is able to solve these problems in the worst case in 
polynomial time, leading to the famous P-NP ■problem. 
One way to try to understand the root of the apparent 
computational hardness is to analyze hard instances of 
problems. This has attracted much interest in statisti- 
cal physics Phase transitions on suitably chosen 
ensembles of random instances were found, e.g., for the 
Satisfiability Problem [6(, the Traveling Salesman Prob- 
lem Q or the vertex-cover problem JVC) Q. For ex- 
act branch-and-bound algorithms 0, [l(| the hardest in- 
stances are found right at these phase transitions, of- 
ten related to a change from a typically polynomially 
("easy") to a typically exponentially ("hard") region. 
Branch-and-bound algorithms systematically explore the 
space of feasible solutions (branching) while trying to 
avoid uninteresting configurations via updating efficient 
bounds. The behavior of these exact algorithms can be 
partially understood in terms of an effective dynamics 
inside the phase diagrams [HI, EH • In practice very effi- 
cient but not exact are stochastic algorithms, e.g. Walk- 
SAT Il3| or ASAT [14J and message-passing algorithms 



|15l Il6j . inspired by statistical mechanics methods like 
the cavity approach [l7| . Also these types of algorithms 
rely on either moving in configuration space or on cal- 
culating iteratively probabilities (weights) for different 
subspaces of configurations. 

Here, we consider a completely different and comple- 
mentary type of algorithm, linear programming (LP), 
which is a standard approach for practical optimization 
problems Q. In connection with cutting planes (CP) 
18], it is a very efficient (but apparently still worst- 
case exponential) approach to combinatorial optimiza- 
tion problems. This approach is fundamentally different 
from the algorithms mentioned above since it does not 
move inside the configuration space but instead consid- 



ers non-feasible (non-combinatorial) assignments to the 
variables which are always more optimal than the true 
feasible solution. Cutting planes are constraints which 
are added additionally and iteratively to the problem un- 
til a feasible solution is found, which is then the optimal 
solution. In particular we study the vertex cover problem 
via LP and CP for Erdos-Renyi random graphs [19j . We 
show that VC with our LP / CP implementation changes 
from "easy" to "hard" right at the same transition point, 
where this change occurs for a branch-and-bound algo- 
rithm, and where the solution landscape changes from 
simple (replica symmetric in the spin- glass language 17] ) 
to complex (replica-symmetry broken). Hence, our re- 
sults indicate that the typical hardness of a problem 
seems to be quite universal since the changes from "easy" 
to "hard" are visible for algorithms which are based on 
fundamentally different notions of configuration space. 

Model Let G = (V, E) be an undirected graph with 
N vertices i G V and M edges {i,j} 6 E. A vertex 
cover Wc C V is a subset of vertices so that for all edges 
{i,j} S E at least one end i or j is contained in Vyc- 
The vertices i S Vyc are called covered, uncovered else. 
We are interested in vertex covers of G of minimum car- 
dinality |Vvc|: the minimum vertex covers. The decision 
problem if a VC with fixed cardinality exists or not be- 
longs to the class of NP-complete problems [l| . 

The analytical solution of VC on Erdos-Renyi graphs 
exhibits a phase transition at the average connectivity 
c = e w 2.7183: for c < e, the solution is replica symmet- 
ric, while for c > e replica symmetry breaking was found 
Q . This can be seen also numerically when clustering the 
minimum vertex covers 1201. Furthermore, in connection 



with the leaf-removal heuristic [21|, the typical-case com- 
plexity of a branch-and-bound algorithm changes form 
"easy" to "hard" at c = e. 

Linear- Programming Approach First, we translate 
the VC problem to an integer linear programming (ILP) 
problem Q, each of the N nodes of the graph is rep- 
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resented by a variable Xi <E {0,1}, i = l,...,N. The 
value Xi — 1 denotes a covered, X{ = indicates an un- 
covered node. The fact that for each edge {i,j} i or j 



must be covered can be written as 



> 1. Min- 



imizing the cardinality of the cover means we want to 
minimize Y] { Xj. When we relax the integer constraint to 
Xi E [0,1], the set of constraints Xi + x j > 1 describes 
a polytope. Now we obtain the following linear program- 
ming problem (LP): 



Minimize 
Subject to 



X — 2^i=l x i 

0<Xi<lVieV 
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This can be solved efficiently, i.e., typically in polyno- 
mial time, by the simplex algorithm (SX) 0, Ea ]. We 
used the public available lp_solve [23| with Bland's first 
index pivoting [2J| . Note that now the solutions are not 
guaranteed to be integer-valued any more, variables with 
x.i G ]0, 1[ we call undecided. Such solutions we call in- 
complete. The value of x = J2i x i is always a lower bound 
for the cardinality of a complete solution. On the other 
hand, if a solution computed by SX is complete, i.e., all 
variables are integer-valued, then it is immediately clear 
that it is a correct minimum for VC. 

Cutting-Plane Approach In order to obtain more 
complete solutions, the CP approach [3] can be used. 
The basic idea is to limit the solution space by adding 
extra constraints, which exclude incomplete solutions. In 
principle, many types of extra constraints are possible. 
Here, we apply the following heuristics, inspired by the 
nature of the problem: For any graph which is a cycle, 
setting all variables of non- isolated nodes to 0.5 (0 else) 
is a solution of the LP, but incomplete. Nevertheless, for 
a cycle of odd length I = 2k + 1, the size of the minimum 
cover is always fc + 1 > k + 0.5. Hence, for any cycle of 
length I = 2k + 1 in a graph, at least fc + 1 nodes of the 
cycle must be covered. Thus, after an execution of SX, if 
the solution is incomplete, we try to detect cycles of odd 
length I where the condition 
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is violated ( [Z/2] is the largest integer larger or equal 
to 1/2) and add this constraint to the LP. Technically, 
the loops are obtained by searching a random spanning 
tree (ST) in the graph via a random breadth-first search 
and adding a randomly chosen edge, which is part of 
the graph but not of the ST. Our algorithm stops, if 
s = 20M times for a randomly chosen spanning trees 
and for all loops emerging from these trees we did not add 
a constraint (because the loop was of even length or the 
constraint was already fulfilled by the current incomplete 
solution). Otherwise, SX is executed for the next time 
and the solution checked for completeness again. 



In general, SX guarantees to obtain a solution in a cor- 
ner of the polytope with minimum x. This means that 
still non-integer solutions can be obtained, how frequently 
depends also on the heuristics used in the actual SX im- 
plementation. Anyway, an incomplete solution obtained 
by CP+SX provides another lower bound x, usually bet- 
ter (but never worse) than that obtained by SX alone. 

Node heuristics To complete an incomplete solution, 
we also applied the following "node" heuristics (NH): It 
randomly selects a vertex i with an undecided variable 
x,i E ]0, 1[ and adds a;, = to the LP and solves it again. 
This forces the SX algorithm to set nodes j adjacent to i 
to Xj = 1. After each run of the SX algorithm we checked 
whether still undecided variables are found and if neces- 
sary the procedure is repeated. This ensures that finally 
a complete solution, i.e., a vertex cover is found, but it 
doesn't have to be a minimum one. Hence the values of 
x obtained in this way are upper bounds to the true min- 
imum vertex covers. Note that we also tried a heuristics 
where Xj = 1 is added to the LP, but it provided typi- 
cally higher values of x as solution, in particular for large 
graph connectivity c > 5. 

Results Next, simulation results for the different 
types of algorithms are presented for ER random graphs 
of N nodes, up to N = 280. We used the ensemble where 
for each graph M edges are created randomly with uni- 
form weight, i.e., the connectivity is c = 2M/N . 

Fig. [1] shows the fraction pf of graphs which exhibit a 
complete solution, for the SX (inset) and SX+CP algo- 
rithms, obtained from averaging over 1000 realizations of 
graphs and for different system sizes, respectively. Ap- 
parently SX is able to find solutions up to about c = 1, 
where a sharp drop of pf is visible, resembling a phase 
transition. Note that the percolation transition of the ER 
ensemble is at c = 1 . For c < 1 , ER random graphs con- 
sist mainly of trees, which are apparently easy to solve, 
even for SX. When including CP, more samples can be 
solved, the transition shifts to a point close to the c = e, 
where replica symmetry breaking occurs and where the 
exact configuration-space-based branch-and-bound algo- 
rithm (with leaf removal |2l|) starts to exhibit a typically 
exponential running time. 

An indicator for the running time of the SX+CP algo- 
rithm is the average number of extra constraints M cxtra 
that were added to the LP resulting from CPs (P) to 
obtain complete solutions. Fig. [5] shows M extra /7V as a 
function of connectivity c. Clearly, an increase close to 
c = e is visible. Note that M extra /7V increases also for all 
realizations (see inset), but in this case this is less infor- 
mative, since the algorithm is stopped if for some time 
no new constraint could be added. 

Finally, Fig. [3] shows the phase diagram for the VC 
problem. In addition to the exact minimum VC and the 
analytical solution [8], simulation results for the SX and 
the SX+CP algorithm (lower bounds) and a combination 
of SX/SX+CP with NH (upper bounds) are included. 
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FIG. 1: Fraction pf of complete solutions for CP approach 
as a function of connectivity c for ER random graphs. The 
inset shows Pf for the SX algorithm. The vertical line denotes 
c — e, the other lines are guides to the eyes only. 
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FIG. 2: Average number of extra constraints M oxtra per node 
for CP algorithm as a function of c only for complete solutions 
and for all solutions (inset). The vertical line denotes c — e, 
the other lines are guides to the eyes only. 



These bounds were obtained, respectively by averaging 
the "cover size" x for different system sizes N and dif- 
ferent connectivities c, yielding x(N, c). We extrapolated 
to infinite system sizes x{c) = liniAr^oo x(N, c) via fitting 
the data to functions x(N, c) = x(c) + a N~ b (see inset of 
figure O or x(N,c) = [x c (c) + a N~ b ]- [1 + / N~ 9 ]. The 
latter was only used for the SX+CP/SX+CP+NH ap- 



proaches for c > 5, where apparently stronger finite-size 
corrections occur. We found, e.g., for SX+CP at c = 4 a 
value b — 0.88(9) which is compatible with the scaling of 
the exact value obtained by branch-and-bound [251 ] . 



covered 




0.53 

T SX+NH — b— 

fSX+CP+NH e o.51 

/ min. VC A 

analytic sol. 

SX+CP 

SX o 



c = 4.0 



100 200 300 

N 



0.7 
0.6 
0.5 
~ 0.4 
0.3 
0.2 
0.1 


0123456789 10 11 

c 

FIG. 3: Phase diagram for the fraction of covered vertices 
x. Minimum VC found with exact Branch-and-Bound algo- 
rithm/analytics [8| as well as upper and lower bounds ob- 
tained from the different algorithms. Inset: Finite-size scaling 
for SX+CP and c = 4.0 The vertical line denotes c = e, dot- 
ted line in the inset a result from the fit (0.52 + 0.40JV -0 - 88 ) 
and the other lines are guides to the eyes only. 

The SX algorithm alone only yields results close to the 
minimum VC up to c = e. Above this value the critical 
fraction of covered vertices converges towards the trivial 
solution x c = 0.5. The SX+CP approach results in a bet- 
ter lower bound and deviates only for an average degree 
c > 5 visibly from the true minimum cover sizes. Com- 
paring with the previous results, this means for c > e, 
this approach typically does not yield the correct solu- 
tion but comes very close to it. The upper bounds also 
start to deviate significantly for c > e, but stay rather 
close. In general, one sees again the importance of the 
critical line c = e: For smaller connectivities c all bounds 
seem to agree but beyond it they start to diverge, which 
is in contrast to previous analytical bounds [3| , which do 
not match the correct result for all connectivities c > 0. 

Conclusion/ Outlook We studied the vertex-cover 
problem for Erdos-Renyi random graphs with a linear 
programming/cutting plane algorithm. The algorithm 
shows a clear "easy-hard" signature close to the con- 
nectivity c = e. This measn that this point denotes a 
phase transition not only for configuration-space-based 
quantities and algorithms 0, EH but also for the LP/CP 
approach which operates outside the space of feasible so- 
lutions. Thus, the typical hardness of VC is really an 
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intrinsic property of the problem and not bounded to 
specific algorithms. This finding may be related to the 
fact that also in the worst-case, all algorithms known for 
NP-hard problems exhibit an exponential running time, 
i.e. it could help in order to understand better the P-NP 
problem. 

In principle, the number of loops grows exponentially 
with the number of nodes. Hence, one could imagine 
that even within our SX+CP approach, by exhausting 
the set of these loop constraints, one arrives at an (ex- 
ponentially slow) but complete algorithm. Nevertheless, 
there are graphs, where our constraints are clearly in- 
sufficient: A simple example is a complete graph of size 
TV = 4, i.e., where each node is connected to any other 
node. The result of SX will be to set all xi = 0.5, i.e., 
x = 2. By adding constraints for the four possible loops 
of odd length, the result will be Xi — 2/3 for all nodes, 
i.e., x = 8/3, while the correct minimum-cover size is 
x = N - 1 = 3. 

One could improve the algorithm in principle by adding 
other types of constraints, e.g., general Gomory-Chvdtal 
cuts [18j. Alternatively, one could consider small sub- 
graphs G' = (V',E'), V C V and E' c E, solve them 
by an exact algorithm yielding the cardinality X' of the 
minimum cover. Then one could add the constraint 
Siey x i — X 1 . We have performed some preliminary 
experiments with these types of cuts, but observed only 
marginal improvements so far, i.e., the overall behavior 
with the transition close to c = e was preserved. 

In practice often a combination of the branching, i.e. 
configuration-space based, and cutting, i.e., LP-based ap- 
proaches, are used. Here, branching sets in when all 
available cutting planes are exhausted. Thus, it would 
be very interesting to see how this combination of ap- 
proaches performs on VC for ER graphs. 

Furthermore, it would be of high interest to analyti- 
cally analyze the cutting plane approach, to see whether 
one can understand why it performs so well for c < e. 
This would lead to a better understanding of the roots 
of computational hardness and could also lead to refined 
bounds on the minimum cover sizes, providing techniques 
applicable to a vast range of problems. 

Finally, it could be worthwhile to extend the present 
study to other graph ensembles. One could test whether 
at the same point replica symmetry breaking occurs, hi- 
erarchical clustering of solution space can be found, and 
the problem becomes hard for configuration-space-based 
branch-and-bound approaches as well as for the LP-based 
cutting-plane algorithms. 
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